Image processing apparatus, method, and program that classifies data of images

ABSTRACT

In an image processing apparatus, a face detection unit  61  detects a face region in a target image. An age recognition unit  62  recognizes the age of a subject based on data of the face region. A similar image search unit  63  searches for data of an existing image having a face region similar to the detected face region as similar image data. A date comparing unit  65 , when the age of the subject is confirmed to be within a first range by an age confirmation unit  64 , acquires a time difference between the capture dates of the target image data and the similar image data and compares the time difference with a second range. A grouping unit  66 , when the time difference is within the second range, classifies the target image data into the same group as the group to which the similar image data belongs.

The present application is based on and claims the benefit of priorityfrom Japanese Patent Application No. 2010-149665 filed on Jun. 30, 2010,the content of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to technology that can classify data ofimages respectively including different subjects having similar faces,such as parent and child or siblings, into different groups with highaccuracy.

2. Related Art

Recently, memory storage sizes are increasing, and digital cameras havebecome capable of storing data of an extremely large number of images.

In such cases, with the increase in the number of stored images,operations of searching for data of a desired image and organizing dataof images that have been searched for is becoming difficult.

In order to reduce such operational difficulty, there is known atechnique that extracts a region of a face (hereinafter, referred to as“face region”) included in an image by way of face detection technology,associates data of the image from which a face is detected with aspecific group, and thereby classifies data of a plurality of imagesinto groups of respective subjects.

However, since data of different images from which identical or similarface regions are detected is classified into the same group, there maybe cases in which a group assigned to a predetermined person happens toinclude image data of a different person having a similar face to theperson.

For example, consideration is given to a case of classifying data of afirst image including a predetermined person as a subject and data of asecond image including a different person as a subject, who has aparent-child or sibling relationship with the predetermined person.

In this case, if both the first and second images are photographed atthe same time, e.g., at the present time, since the two subjects aredifferent in age, it is possible to classify data thereof into differentgroups. This is because, when comparing a first face region detectedfrom the first image and a second face region detected from the secondimage, there are differences in the features due to age differencetherebetween, more specifically, horizontal to vertical ratio of thefaces, height of eyes on the faces, size of eyes, number of wrinkles, orrecession of hair. Therefore, it is possible to classify data of thefirst and second images into different groups according to difference insuch features.

However, in a case in which the predetermined person is, for example, aparent, the subject included in the first image is the parent at the ageof 4, the different person is a child, and the subject included in thesecond image is the child at the age of 4, then it becomes difficult toclassify data of such images into different groups. This is because thetwo subjects are the same in age, and there are no differences infeatures in terms of age difference as described above. Furthermore,faces of a parent and a child at the same age are most likely verysimilar. Therefore, in such a case, the respective data of the first andsecond images cannot be separated, and the first and second images maybe classified into the same group of either the parent or the child.

For this reason, it is desired to realize a method capable ofclassifying data of images respectively including different subjectshaving similar faces such as parent and child or siblings into differentgroups at high accuracy.

SUMMARY OF THE INVENTION

It is an object of the present invention to classify data of imagesrespectively including different subjects having similar faces intodifferent groups, with high accuracy.

In order to attain the above described object, in accordance with afirst aspect of the present invention, there is provided an imageprocessing apparatus comprising: a face detection unit that detects aregion including a face of a subject in a target image, as a faceregion, using data of a captured image that can identify a capture datethereof, as the data of the target image; an age recognition unit thatrecognizes the age of the subject in the target image based on the faceregion detected by the face detection unit; a similar image search unitthat searches for data of an existing image having a face region similarto the face region detected by the face detection unit, as data of asimilar image from among data of a plurality of existing images that canrespectively identify capture dates thereof, each of which belongs toone of a plurality of groups; an age confirmation unit that confirmswhether or not the age of the subject in the target image recognized bythe age recognition unit is within a first range; a date comparing unitthat, in a case in which the age confirmation unit confirms that the ageof the subject in the target image is within the first range, acquires atime difference between a first capture date of the data of the targetimage and a second capture date of the data of the similar image foundby the similar image search unit, and compares the time difference witha second range; and a grouping unit that classifies the data of thetarget image into the same group as the group to which the data of thesimilar image belongs, in a case in which the time difference is withinthe second range, as a result of a comparison by the date comparingunit.

In order to attain the above described object, in accordance with asecond aspect of the present invention, there is provided an imageprocessing method carried out by an image processing apparatus forclassifying data of a captured image that can identify a capture datethereof, the method comprising the steps of: detecting a regionincluding a face of a subject in a target image, as a face region, usingdata of a captured image that can identify a capture date thereof, asthe data of the target image; recognizing age of the subject in thetarget image based on the face region thus detected; searching for dataof an existing image having a face region similar to the face regionthus detected, as data of a similar image, from among data of aplurality of existing images that can respectively identify capturedates thereof, each of which belongs to one of a plurality of groups;confirming whether or not the age of the subject in the target imagethus recognized is within a first range; acquiring, in a case in whichit is confirmed that the age of the subject in the target image iswithin the first range, a difference between a first capture date of thedata of the target image and a second capture date of the data of thesimilar image thus found, and comparing the difference with a secondrange; and classifying the data of the target image into the same groupas the group to which the data of the similar image belongs, in a casein which the difference is within the second range, as a result of a thecomparison.

In order to attain the above described object, in accordance with athird aspect of the present invention, there is provided anon-transitory storage medium having stored therein a program executableby a computer that controls an image processing apparatus that carriesout image processing on data of a captured image that can identify acapture date thereof, causing the computer to realize functions of:detecting a region including a face of a subject in a target image, as aface region, using data of the captured image that can identify acapture date thereof as the data of the target image; recognizing age ofthe subject in the target image based on the face region thus detected;searching for data of an existing image having a face region similar tothe face region thus detected, as data of a similar image, from amongdata of a plurality of existing images that can respectively identifycapture dates thereof, each of which belongs to one of a plurality ofgroups; confirming whether or not the age of the subject in the targetimage thus recognized is within a first range; acquiring, in a case inwhich it is confirmed that the age of the subject in the target image iswithin the first range, a difference between a first capture date of thedata of the target image and a second capture date of the data of thesimilar image thus found, and comparing the difference with a secondrange; and classifying the data of the target image into the same groupas the group to which the data of the similar image belongs, in a casein which the difference is within the second range, as a result of thecomparison.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a hardware configuration of an imageprocessing apparatus according to one embodiment of the presentinvention;

FIG. 2 is a block diagram showing a functional configuration of theimage processing apparatus shown in FIG. 1 to carry out groupingdetermination processing;

FIG. 3 is a flowchart showing flow of the grouping determinationprocessing carried out by the image processing apparatus shown in FIG.2; and

FIG. 4 is a diagram illustrating an outline of images grouped by animage processing unit of the image processing apparatus shown in FIG. 2.

DETAILED DESCRIPTION OF THE INVENTION

The following describes an embodiment of the present invention withreference to the drawings.

FIG. 1 is a block diagram showing a hardware configuration of an imagecapturing apparatus 1 as one embodiment of the image processingapparatus according to the present invention. The image capturingapparatus 1 can be configured by a digital camera, for example.

The image capturing apparatus 1 is provided with a CPU (CentralProcessing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random AccessMemory) 13, a bus 14, an input/output interface 15, an image capturingunit 16, an operation unit 17, a display unit 18, a storing unit 19, acommunication unit 20, and a drive 21.

The CPU 11 executes various processes according to programs that arestored in the ROM 12. Alternatively, the CPU 11 executes variousprocesses according to programs that are loaded from the storing unit 19to the RAM 13.

The RAM 13 also stores data and the like, necessary for the CPU 11 toexecute the various processes, as appropriate.

For example, according to the present embodiment, programs forimplementing functions of a face detection unit 61, an age recognitionunit 62, a similar image search unit 63, an age confirmation unit 64, adate comparing unit 65, and a grouping unit 66 shown in FIG. 2, whichwill be described later, are stored in the ROM 12 or the storing unit19. Therefore, each of the functions of the face detection unit 61, theage recognition unit 62, the similar image search unit 63, the ageconfirmation unit 64, the date comparing unit 65, and the grouping unit66 can be realized by the CPU 11 executing processes according to theseprograms.

The CPU 11, the ROM 12, and the RAM 13 are connected to one another viathe bus 14. The bus 14 is also connected with the input/output interface15. The image capturing unit 16, the operation unit 17, the display unit18, the storing unit 19, and the communication unit 20 are connected tothe input/output interface 15.

The image capturing unit 16 is provided with an optical lens unit and animage sensor, which are not illustrated in the drawings.

The optical lens unit is configured by a light condensing lens such as afocus lens, a zoom lens, and the like, for example, to photograph asubject.

The focus lens is a lens that forms an image of a subject on the lightreceiving surface of the image sensor. The zoom lens is a lens forfreely changing a focal point within a predetermined range.

The optical lens unit includes peripheral circuits that adjustparameters such as focus, exposure, white balance, and the like, asnecessary.

The image sensor is configured by an optoelectronic conversion device,an AFE (Analog Front End), and the like.

The optoelectronic conversion device is configured by a CMOS(Complementary Metal Oxide Semiconductor) type optoelectronic conversiondevice, or the like, for example. An image of a subject is made incidentthrough the optical lens unit on the optoelectronic conversion device.The optoelectronic conversion device optoelectronically converts (i.e.,captures) an image of a subject as an image signal at a predeterminedinterval, stores the image signal thus converted, and sequentiallysupplies the stored image signal to the AFE as an analog signal.

The AFE executes various kinds of signal processing such as A/D(Analog/Digital) conversion on the analog image signal. As a result ofthe various kinds of signal processing, a digital signal is generatedand outputted as an output signal from the image capturing unit 16.

Hereinafter, the output signal from the image capturing unit 16 isreferred to as “data of a captured image”. Thus, data of a capturedimage is outputted from the image capturing unit 16 and provided asappropriate to the CPU 11 and the like.

The operation unit 17 is configured by various buttons and receivesoperation instruction from a user.

The display unit 18 is configured by a liquid crystal display and thelike and displays various images.

The storing unit 19 is configured by a DRAM (Dynamic Random AccessMemory) and the like and temporarily stores data of captured imagesoutputted from the image capturing unit 16. Also, the storing unit 19stores various kinds of data necessary for various kinds of imageprocessing, such as image data, values of various flags, thresholdvalues, and the like.

The communication unit 20 controls communication with other devices (notshown) via networks including the Internet.

The input/output interface 15 is connected with the drive 21 asnecessary. Removable media 31 such as a magnetic disk, an optical disk,a magneto-optical disk, or a semiconductor memory is mounted to thedrive 21 as appropriate. Also, programs read from the removable media 31via the drive 21 are installed in the storing unit 19 as necessary.Furthermore, similar to the storing unit 19, the removable media 31 canstore various kinds of data such as image data and the like, stored inthe storing unit 19.

The image capturing apparatus 1 having such a configuration can carryout the following series of processes.

The image capturing apparatus 1 detects a region including a face of asubject as a face region from an image (hereinafter, referred to as“target image”), using data of an image to be grouped.

The image capturing apparatus 1 recognizes the age of the subject basedon data of the face region.

The image capturing apparatus 1 extracts, from among data of images(hereinafter, referred to as “existing image”) that are alreadyclassified into predetermined groups, data of an existing image having aface region similar to the face region that has been detected.

In a case in which the age of the subject in the target image is withina predetermined range and a difference between capture dates of thetarget image data and the existing image data extracted above is withina predetermined range, the image capturing apparatus 1 classifies thetarget image data into the same group as the existing image data.Otherwise, the image capturing apparatus 1 classifies the target imagedata into a group other than the existing image data.

Such a series of processes is hereinafter referred to as “groupingdetermination processing”.

FIG. 2 is a block diagram showing a functional configuration of theimage capturing apparatus 1 that carries out the grouping determinationprocessing.

As shown in FIG. 2, the image capturing apparatus 1 is provided with animage storing unit 41, and an image processing unit 42.

The image storing unit 41 includes a target image storing unit 51, andan existing image storing unit 52.

The image processing unit 42 includes a face detection unit 61, an agerecognition unit 62, a similar image search unit 63, an age confirmationunit 64, a date comparing unit 65, and a grouping unit 66.

In the present embodiment, the image storing unit 41, i.e., the targetimage storing unit 51 and the existing image storing unit 52 areconfigured as an area in the RAM 13, the storing unit 19, or theremovable media 31, from among the constituent elements shown in FIG. 1.

The target image storing unit 51 stores image data acquired by the imagecapturing unit 16 or the like that captures a subject as target imagedata. At this time, the target image storing unit 51 stores metainformation indicating a capture date of the target image data, inassociation with the target image data.

More specifically, in the present embodiment, the target image data isincluded in an EXIF (Exchangeable Image File Format) file. Such a fileincluding target image data is hereinafter referred to as a “targetimage file”. The target image file may include, as well as target imagedata, various kinds of meta information related to the target imagedata. Therefore, in the present embodiment, date information at the timeof capturing is included in the target image file as meta informationindicating the capture date. Such a target image file is stored in thetarget image storing unit 51.

The image data stored as target image data in the target image storingunit 51 is not limited to image data captured by the image capturingunit 16 but may be any kind of image data so long as a capture datethereof is recognizable, and may include, for example, image dataacquired from outside via the communication unit 20.

The existing image storing unit 52 stores data of a plurality of imagesclassified to respective groups according to person and age in advance,as existing image data, in such a group structure. Also, the existingimage storing unit 52 stores meta information indicating the capturedate of the existing image data associated therewith.

More specifically, in the present embodiment, the existing image data isin EXIF format as well as the target image data. Such an EXIF fileincluding the existing image data and meta information indicating thecapture dates thereof is, hereinafter, referred to as “existing imagefile” in order to distinguish it from a target image file.

Compared to such an image storing unit 41, each of the constituentelements of the image processing unit 42, i.e., each of the facedetection unit 61, the age recognition unit 62, the similar image searchunit 63, the age confirmation unit 64, the date comparing unit 65, andthe grouping unit 66 is configured as a combination of the CPU 11 ashardware, and programs stored in the ROM 12 and the like as software,from among the constituent elements shown in FIG. 1.

The face detection unit 61 reads target image data from the target imagestoring unit 51 at predetermined timing such as, for example, the timeof a grouping instruction operation, which will be described later, anddetects a region including a face of a subject in the target image as aface region.

More specifically, in the present embodiment, the face detection unit 61first extracts characteristic points of a face such as end points ofeyebrows, eyes, nose, and mouth, contour points of the face, the toppoint of the head, and the bottom point of the chin, from the targetimage data by way of the characteristic points extraction processingdisclosed in Japanese Patent Application Publication No. 2001-16573 andthe like. The face detection unit 61 determines regions of eyebrows,eyes, nose, and mouth and the boundary thereof as a contour, acquiresposition information thereof, and thereby detects a face region.

A method of detecting a face region applicable to the face detectionunit 61 is not limited to the present embodiment, but any kind of methodthat is already in the public domain or that will be developed in thefuture may be used.

The face detection unit 61 outputs a result of detecting a face regionand the target image data to the age recognition unit 62. Here, as thedetection result of the face region, the face detection unit 61 outputsinformation that can identify the face region from the target image, forexample, or more specifically, position information that can identifythe position of the face region, for example.

Furthermore, the face detection unit 61 outputs the target image dataand information of the capture date thereof to the similar image searchunit 63.

In a case in which no face region is detected, for example, because nosubject is included in the target image, the face detection unit 61supplies the target image data and capture date information thereoftogether with information that no face region is detected to thegrouping unit 66 via the age recognition unit 62 and the ageconfirmation unit 64.

The age recognition unit 62 recognizes the age of the subject includedin the target image based on data of the face region in the target imagedetected by the face detection unit 61.

More specifically, in the present embodiment, the age recognition unit62 first detects data indicative of features of face (hereinafter,referred to as “face feature data”) by way of age presumption processingdisclosed by Japanese Patent Application Publication No. 1994-333023.The face feature data may include, for example, data of horizontal tovertical ratio of face, height of eyes on face, size of eyes, number ofwrinkles, recession of hair, and the like. Next, the age recognitionunit 62 generates data of a renewed face region by combining patterndata of existing face parts based on the face feature data andrecognizes the age of the subject based on the data of the renewed faceregion.

A method of age recognition applicable to the age recognition unit 62 isnot limited to the method of the present embodiment, but any kind ofmethod that is already public domain or that will be developed in thefuture may be used.

The age recognition unit 62 outputs the target image data and ageinformation thus recognized to the age confirmation unit 64.

The similar image search unit 63 searches for data of an existing image(hereinafter, referred to as “similar image”) having a face regionsimilar to the face region of the target image detected by the facedetection unit 61 from among data of a plurality of existing imagesstored in the existing image storing unit 52.

Here, as a method of determining whether or not face regions are similarto each other, a method is employed in the present embodiment such that,computing similarity for each of a plurality of features in the faceregions and using overall similarity of the plurality of features, it isdetermined whether or not the face regions are similar to each other.However, the method of determining whether or not the face regions aresimilar to each other is not limited to the method of the presentembodiment, and any kind of method that is already in the public domainor that will be developed in the future may be used.

The similar image search unit 63 supplies the search result of similarimage data to the age confirmation unit 64.

There are cases, details of which will be described later, where the ageconfirmation unit 64, upon receiving the search result, issues a requestfor acquisition of information (hereinafter, referred to as “similarimage information”) indicative of the capture date of the similar imagedata and the group to which the similar image data belongs.

In this case, the similar image search unit 63 acquires the similarimage information from the existing image storing unit 52 and outputs italong with the target image data and information of the capture datethereof.

In a case in which similar image data is found (the search result tothat effect is received) by the similar image search unit 63, the ageconfirmation unit 64 confirms whether or not the age of the subject inthe target image recognized by the age recognition unit 62 falls withina predetermined range (hereinafter, referred to as “target age range”).

More specifically, in the present embodiment, it is assumed that thetarget age range is between 0 and 5 years old. That is, the ageconfirmation unit 64 confirms whether or not the age of the subject inthe target image is within the range of 0 to 5 years old, i.e., the agedoes not exceed 5 years old.

In a case in which it is confirmed that the age of the subject in thetarget image is within the target age range (0 to 5 years old), the ageconfirmation unit 64 issues an acquisition request for the similar imageinformation to the similar image search unit 63, as described above.When the similar image search unit 63 that has received the acquisitionrequest outputs the similar image information, i.e., informationindicating the group to which the similar image data belongs and thecapture date thereof, the age confirmation unit 64 acquires the similarimage information. Then, the age confirmation unit 64 outputs the targetimage data, the capture date information thereof, and the similar imageinformation (information indicating the capture date of the similarimage data and the group thereof acquired from the similar image searchunit 63) to the date comparing unit 65.

On the other hand, in a case in which it is confirmed that the age ofthe subject in the target image is outside the target age range (0 to 5years old), the age confirmation unit 64 outputs the target image dataand the capture date information thereof to the grouping unit 66.

Moreover, in a case in which no similar image is found by the similarimage search unit 63 (the search result to that effect is received), theage confirmation unit 64 outputs the target image data and the capturedate information thereof to the grouping unit 66 without carrying outthe processing of confirming age.

In the case in which the age of the subject in the target image iswithin the target age range (0 to 5 years old), the age confirmationunit 64 provides the target image data, the capture date thereof, andthe similar image information (information indicating the capture dateand the group of the similar image data) to the date comparing unit 65.

Then, the date comparing unit 65 compares the capture date of the targetimage data with that of the similar image data.

More specifically, in the present embodiment, the date comparing unit 65computes time difference between the capture dates of the target imagedata and the similar image data and determines whether or not the timedifference is within a specified time interval. Here, the specified timeinterval refers to a time interval arbitrarily specifiable by a user'soperation on the operation unit 17.

When it is determined that the time difference between the capture datesof the target image data and the similar image data is within thespecified time interval, the date comparing unit 65 outputs to thegrouping unit 66 the target image data and the information of the groupto which the data of the similar image to the target image belongs.

On the other hand, when it is determined that the time differencebetween capture dates of the target image data and the similar imagedata exceeds the specified time interval, the date comparing unit 65compares the capture dates of the target image data and another similarimage data. Such a comparing process by the date comparing unit 65 isrepeated until data of a similar image is found such that the timedifference between capture dates falls within the specified timeinterval, or until no more similar image data is found.

The grouping unit 66 classifies target image data into a predeterminedgroup and stores the target image data in the existing image storingunit 52 as data belonging to the classified group.

More specifically, in the present embodiment, in the case in which thetime difference between capture dates of the target image data and thesimilar image data is within the specified time interval, the datecomparing unit 65 provides to the grouping unit 66 the target image dataand the information of the group to which the similar image data thereofbelongs. In this case, the grouping unit 66 classifies the target imagedata into the group to which the similar image data belongs by attachingthe same tag information to the target image data as the similar imagedata, for example. The grouping unit 66 stores in the existing imagestoring unit 52 the target image data as data belonging to the group.

Processing in other cases will be described later with reference to theflowchart of FIG. 3.

In the following, the grouping determination processing carried out bythe image capturing apparatus 1 shown in FIG. 2 will be described withreference to FIG. 3.

FIG. 3 is a flowchart showing flow of the grouping determinationprocessing.

The grouping determination processing starts with a user's instructionoperation on the operation unit 17 to classify target image data, forexample, and the following processing is carried out.

In step S11 of FIG. 3, the face detection unit 61 reads target imagedata from the target image storing unit 51 and attempts to detect a faceregion therefrom.

In step S12, the grouping unit 66 determines whether or not a faceregion is detected from the target image.

In a case in which no face region is detected by the face detection unit61 due to a reason such as that no subject is included in the targetimage, NO is determined in step S12, and the grouping determinationprocessing ends. In this case, the target image data is stored in theexisting image storing unit 52 without being classified to any group.

On the other hand, in a case in which a face region is detected by theface detection unit 61, YES is determined in step S12, and controlproceeds to step S13.

In step S13, the age recognition unit 62 recognizes the age of thesubject in the target image based on the face region data detected bythe face detection unit 61.

In step S14, the similar image search unit 63 searches for data of anexisting image having a face region similar to the face region detectedby the face detection unit 61 as data of a similar image to the targetimage.

In step S15, the grouping unit 66 determines whether or not any similarimage is found.

In a case in which no similar image is found, NO is determined in stepS15, and the grouping determination processing ends. In this case, thedata of the target image is stored in the existing image storing unit 52without being classified to any group.

On the other hand, in a case in which one or more similar images arefound by the similar image search unit 63, YES is determined in stepS15, and control proceeds to step S16.

In step S16, the age confirmation unit 64 determines whether or not theage of the subject in the target image recognized by the age recognitionunit 62 is within a predetermined target age range.

The target age range is an age range, which can be specified by a useroperating the operation unit 17 before carrying out the groupingdetermination processing, as described above, and is set to 0 to 5 yearsold in the present embodiment.

In a case in which the age of the subject in the target image is outsidethe target age range, i.e., exceeds 5 years old, NO is determined instep S16, and control proceeds to step S20.

In step S20, the grouping unit 66 classifies the target image data intoa “miscellaneous” group by attaching tag information of “miscellaneous”to the target image data, for example. With this, the groupingdetermination processing ends. In this case, the target image data isstored in the existing image storing unit 52 along with the capture dateinformation thereof as being classified to the miscellaneous group.

On the other hand, in a case in which the age of the subject in thetarget image is within the target age range, i.e., does not exceed 5years old, YES is determined in step S16, and control proceeds to stepS17.

In step S17, the date comparing unit 65 compares the capture date of thesimilar image data found by the similar image search unit 63 with thecapture date of the target image data. More specifically, the datecomparing unit 65 computes a time difference between the capture datesof the target image data and the similar image data.

In step S18, the date comparing unit 65 determines whether or not thetime difference between the capture dates of the target image data andthe similar image data is within the specified time interval.

What is meant by “the time difference between the capture dates of thetarget image data and the similar image data exceeds the specified timeinterval” is that the subjects of the target image and the similar imagehave similar faces beyond a long period over the specified timeinterval, e.g., 2 years. This means that the subjects of the targetimage and the similar image are most likely different people but have aclose relationship such as parent-child or siblings.

Therefore, in the case in which the time difference between the capturedates of the target image data and the similar image data exceeds thespecified time interval, NO is determined in step S18, control goes backto step S14, another similar image is searched for, and the processesthereafter are repeated.

After that, as a result of having repeated the processes of steps S14 toS18 on another similar image, if it happens that the time differencebetween capture dates of the target image data and the similar imagedata is within the specified time interval, then it is determined thatthe subjects of the target image and the similar image have similarfaces for a short period within the specified time interval. This meansthat the subjects of the target image and the similar image can bedetermined to be the same person.

Therefore, in the case in which the time difference between the capturedates of the target image data and the similar image data is within thespecified time interval, YES is determined in step S18, and controlproceeds to step S19.

In step S19, the grouping unit 66 classifies the target image data intothe same group as the similar image data and stores the target imagedata in the existing image storing unit 52 as data belonging to thegroup. More specifically, the grouping unit 66 classifies the targetimage data into the same group as the similar image data by attachingtag information indicating the same group as the similar image data tothe target image data, for example. Then, the grouping unit 66 storesthe target image data in the existing image storing unit 52 as databelonging to the group.

With this, the grouping determination processing ends.

In the following, a more specific description of the groupingdetermination processing will be given, with reference to FIG. 4.

FIG. 4 is a diagram illustrating a specific processing result of thegrouping determination processing.

In the example of FIG. 4, data of target images 71 a, 71 b, 71 c, 71 d,and so forth captured by the image capturing unit 16 is stored in thetarget image storing unit 41. Also, information that can identifycapture dates of the target images, which are shown in upper right ofthe target images 71 a, 71 b, 71 c, and 71 d of FIG. 4, is assumed to bestored in the target image storing unit 51.

Furthermore, it is assumed that the subjects 72 a and 72 c included inthe target images 71 a and 71 c are the same child, while the subject 72b included in the target image 71 b is a parent, and the subject 72 dincluded in the target image 71 d is somebody other than the parent andchild.

Furthermore, it is assumed that a “2006, child, 4 years old” group iscreated in advance in the existing image storing unit 52 to which astoring unit 81 a (hereinafter, referred to as “child group storing unit81 a”) that stores image data belonging to the group is provided inadvance. It is assumed that data of one or more existing imagesincluding a 4 year old child as a subject thereof (only the existingimage 82 a is shown) are stored along with the capture date informationthereof in the child group storing unit 81 a.

Also, it is assumed that a “1986, parent, 4 years old” group is createdin advance in the existing image storing unit 52 to which a storing unit81 b (hereinafter, referred to as “parent group storing unit 81 b”) thatstores image data belonging to the group is provided in advance. It isassumed that data of one or more existing images including a parent at 4years of age as a subject thereof (only the existing image 82 b isshown) is stored along with capture date information thereof in theparent group storing unit 81 b.

Also, a “miscellaneous” group is created in advance in the existingimage storing unit 52 to which a storing unit 81 c (hereinafter,referred to as “miscellaneous group storing unit 81 c”) that storesimage data belonging to the group is provided in advance.

This means that, in the example of FIG. 4, it is assumed that the dataof each of the target images 71 a, 71 b, 71 c, 71 d, and so forth isclassified into one of the three groups, the “2006, child, 4 years old”group, the “1986, parent, 4 years old” group, and the “miscellaneous”group.

In step S11 of FIG. 3, it is assumed that data of the target image 71 ahas been read from the target image storing unit 51, and the facedetection unit 61 has attempted to detect a face region.

In this case, since the face region 73 a is detected as shown in FIG. 4,YES is determined in the process of step S12, and the age recognitionunit 62 recognizes that the age of the subject (child) of the targetimage is 4 years old based on data of the face region 73 a in theprocess of step S13.

In the process of step S14, the similar image search unit 63 searchesfor data of an existing image having a face region similar to the faceregion 73 a as data of a similar image to the target image from amongdata of a plurality of existing images stored in the existing imagestoring unit 52.

Though the target and the order of the search are not limited, for easeof description, it is assumed here that the child group storing unit 81a, the parent group storing unit 81 b, and the miscellaneous groupstoring unit 81 c are searched for as the target of search, in thisorder.

The existing images having face regions similar to the face region 73 aincludes the existing image 82 a, data of which is stored in the childgroup storing unit 81 a, the existing image 82 b, data of which isstored in the parent group storing unit 81 b, and the like.

However, since the child group storing unit 81 a is firstly searchedfor, the data of the existing image 82 a, for example, is to be found asdata of a similar image to the target image 71 a.

In this case, YES is determined in the process of step S15, and the ageconfirmation unit 64 determines whether or not the age (4 years old) ofthe subject (child) of the target image 71 a is within a predeterminedtarget age range in the process of step S16.

Since it has been assumed here that the target age range is set between0 and 5 years old as described above, YES is determined in the processof step S16, and control proceeds to step S17.

In the process of step S17, the date comparing unit 65 computes a timedifference between the capture dates of the data of the target image 71a and the data of the similar image 82 a.

As shown in FIG. 4, the capture date of the data of the target image 71a is March 2006. On the other hand, the capture date of the data of thesimilar image 82 a is, though not explicitly shown in FIG. 4, at leastin the year 2006, since the data of the similar image 82 a belongs tothe “2006, child, 4 years old” group. Therefore, the time difference isat most 9 months. Here, a description will be given assuming that thetime difference is 9 months.

In the process of step S18, the date comparing unit 65 determineswhether or not a time difference between the capture dates of the dataof the target image 71 a and the data of the similar image 82 a iswithin the specified time interval.

Though the specified time interval can be arbitrarily specified by auser, as described above, since it suffices if parent and child aredistinguishable, it is assumed that a considerably long interval of, forexample, 10 years is specified here.

In this case, the time difference of 9 months is obviously shorter thanthe specified time interval of 10 years. This means that the subjects ofthe target image 71 a and the similar image 82 a have similar faceswithin a short period (of 9 months, in this case) not exceeding thespecified time interval, and it can be determined that the subjects ofthe target image 71 a and the similar image 82 a are the same person,i.e., the child.

Thus, YES is determined in the process of step S18, the grouping unit 66classifies the data of the target image 71 a into the “2006, child, 4years old” group and stores it in the child group storing unit 81 a inthe process of step S19.

With this, the grouping determination processing ends.

After that, it is assumed that the grouping determination processingstarts again, the target image 71 c is read from the target imagestoring unit 51, and the face detection unit 61 attempts to detect aface region in the process of step S11.

In this case, the target image 71 c is processed exactly in the same wayas the target image 71 a. That is, in the end, the grouping unit 66classifies the data of the target image 71 c into the “2006, child, 4years old” group and stores it in the child group storing unit 81 a inthe process of step S19.

With this, the grouping determination processing ends.

After that, it is assumed that the grouping determination processingstarts over again, the target image 71 b is read from the target imagestoring unit 51, and the face detection unit 61 attempts to detect aface region in the process of step S11.

In this case, since the face region 73 b is detected as shown in FIG. 4,YES is determined in the process of step S12, and the age recognitionunit 62 recognizes that the age of the subject (parent) of the targetimage is 4 years old based on data of the face region 73 b in theprocess of step S13.

In the process of step S14, the similar image search unit 63 searchesfor data of an existing image having a face region similar to the faceregion 73 b as data of a similar image to the target image from amongdata of a plurality of existing images stored in the existing imagestoring unit 52.

The existing images having face regions similar to the face region 73 bincludes the existing image 82 a, data of which is stored in the childgroup storing unit 81 a, and the existing image 82 b, data of which isstored in the parent group storing unit 81 b.

However, since the child group storing unit 81 a is firstly searchedfor, the data of the existing image 82 a, for example, is to be found asdata of a similar image to the target image 71 b.

In this case, YES is determined in the process of step S15, and the ageconfirmation unit 64 determines whether or not the age (4 years old) ofthe subject (parent) of the target image 71 b is within a predeterminedtarget age range in the process of step S16.

Since it has been assumed here that the target age range is set between0 and 5 years old as described above, YES is determined in the processof step S16, and control proceeds to step S17.

In the process of step S17, the date comparing unit 65 computes a timedifference between the capture dates of the data of the target image 71b and the data of the similar image 82 a.

As shown in FIG. 4, the capture date of the data of the target image 71b is January 1986. On the other hand, the capture date of the data ofthe similar image 82 a is, though not explicitly shown in FIG. 4, atleast in the year 2006, since the data of the similar image 82 a belongsto the “2006, child, 4 years old” group. Therefore, the time differenceis at least 19 years. Here, a description will be given assuming thatthe time difference is 19 years.

In the process of step S18, the date comparing unit 65 determineswhether or not a time difference between the capture dates of the dataof the target image 71 b and the data of the similar image 82 a iswithin the specified time interval.

In this case, the time difference of 19 years is obviously longer thanthe specified time interval of 10 years. This means that the subjects ofthe target image 71 b and the similar image 82 a have similar facesbeyond the period (of 19 years, in this case) over the specified timeinterval of 10 years, and it can be determined that the subject in thetarget image is most likely the parent, while the subject in the similarimage 82 a is the child. In such a case, it is difficult to classify thetarget image data 71 c accurately.

Therefore, in this case, NO is determined in step S18, control goes backto step S14, another similar image is searched for, and the processesthereafter are repeated.

This time, in the process of step S14, it is assumed that the data ofthe existing image 82 b stored in the parent group storing unit 81 b isfound as data of a similar image to the target image 71 b, for example.

In this case, YES is determined in the process of step S15, and the ageconfirmation unit 64 determines whether or not the age (4 years old) ofthe subject (parent) of the target image 71 b is within a predeterminedtarget age range in the process of step S16.

Since it has been assumed here that the target age range is set between0 and 5 years old as described above, YES is determined in the processof step S16, and control proceeds to step S17.

In the process of step S17, the date comparing unit 65 computes a timedifference between the capture dates of the data of the target image 71b and the data of the similar image 82 b.

As shown in FIG. 4, the capture date of the data of the target image 71b is January 1986. On the other hand, the capture date of the data ofthe similar image 82 b is, though not explicitly shown in FIG. 4, atleast in the year 1986, since the data of the similar image 82 b belongsto the “1986, parent, 4 years old” group. Therefore, the time differenceis at most 11 months. Here, a description will be given assuming thatthe time difference is 11 months.

In the process of step S18, the date comparing unit 65 determineswhether or not a time difference between the capture dates of the dataof the target image 71 b and the data of the similar image 82 b iswithin the specified time interval.

In this case, the time difference of 11 months is obviously shorter thanthe specified time interval of 10 years. This means that the subjects ofthe target image 71 b and the similar image 82 b have similar faceswithin a short period (of 11 months, in this case) not exceeding thespecified time interval, and it can be determined that the subjects ofthe target image 71 b and the similar image 82 b are the same person,i.e., the parent.

Thus, YES is determined in the process of step S18, the grouping unit 66classifies the data of the target image 71 b into the “1986, parent, 4years old” group and stores it in the parent group storing unit 81 b inthe process of step S19.

With this, the grouping determination processing ends.

After that, it is assumed that the grouping determination processingstarts over again, the target image 71 d is read from the target imagestoring unit 51, and the face detection unit 61 attempts to detect aface region in the process of step S11.

In this case, since the face region 73 d is detected as shown in FIG. 4,YES is determined in the process of step S12, and the age recognitionunit 62 recognizes that the age of the subject (parent) of the targetimage is 14 years old based on data of the face region 73 d in theprocess of step S13.

In the process of step S14, the similar image search unit 63 searchesfor data of an existing image having a face region similar to the faceregion 73 d as data of a similar image to the target image from amongdata of a plurality of existing images stored in the existing imagestoring unit 52.

Here, though not shown in FIG. 4, it is assumed that the miscellaneousgroup storing unit 81 c stores data of a similar image, and that thedata is found.

In this case, YES is determined in the process of step S15, and the ageconfirmation unit 64 determines whether or not the age (14 years old) ofthe subject (parent) of the target image 71 d is within a predeterminedtarget age range in the process of step S16.

Since it has been assumed here that the target age range is set between0 and 5 years old as described above, NO is determined in the process ofstep S16, and control proceeds to step S20.

In step S20, the grouping unit 66 classifies the data of the targetimage 71 d into the “miscellaneous” group and stores it in themiscellaneous group storing unit 81 c.

With this, the grouping determination processing ends.

As in an embodiment of the present invention described above withreference to FIGS. 1 to 4, an image capturing apparatus 1 is providedwith a face detection unit 61, an age recognition unit 62, a similarimage search unit 63, an age confirmation unit 64, a date comparing unit65, and a grouping unit 66 (See FIG. 2).

The face detection unit 61 detects a region including a subject's facein a target image as a face region using captured image data that canidentify a capture date thereof as the target image data.

The age recognition unit 62 recognizes the age of the subject in thetarget image based on the face region data.

The similar image search unit 63 searches for data of an existing imagehaving a face region similar to the face region detected by the facedetection unit 61 as similar image data from among data of a pluralityof existing images that can respectively identify capture dates thereof,each of which belongs to one of a plurality of groups.

The age confirmation unit 64 confirms whether or not the age of thesubject in the target image recognized by the age recognition unit 62 iswithin a predetermined first range (target age range).

The date comparing unit 65, in a case in which the age of the subject inthe target image is within the first range, acquires a time differencebetween the capture dates of the target image data and the similar imagedata and compares the time difference and a predetermined second range(specified time interval).

As a result of the comparison by the date comparing unit 65, in a casein which the time difference is within the second range, the groupingunit 66 classifies the target image data into the same group as thegroup to which the similar image data belongs.

Here, what is meant by “the time difference is within the second range”is that the subjects of the target image and the similar image havesimilar faces in a short period within the second range. That is, it ispossible to determine that the subjects of the target image and thesimilar image are the same person. Therefore, in such a case, the targetimage data is classified into the same group as the group to which thesimilar image data belongs.

On the other hand, what is meant by “the time difference is outside thesecond range” is that the subjects of the target image and the similarimage have similar faces beyond the period over the second range, e.g.,10 years. That is, the subjects of the target image and the similarimage are most likely different from each other, but in a closerelationship such as parent-child or siblings. In such a case, since itis difficult to classify the target image data accurately, the targetimage data is not classified into the same group as the group to whichthe similar image data belongs.

In this way, it becomes possible to classify data of images respectivelyincluding different subjects having similar faces into different groupswith high accuracy.

It should be noted that the present invention is not limited to theembodiment described above, and modifications and improvements theretowithin a scope in which an object of the present invention can berealized, are included in the invention.

For example, in embodiments described above, although the target imagedata has been described as still image data, the target image data isnot limited to this and can be moving image data. In this case, which isnot illustrated in the drawings, the image capturing apparatus 1 cancarry out the grouping determination processing of FIG. 4 or the like ondata of a predetermined frame including a subject from among frameimages or the like constituting the moving image, or on data of athumbnail image of the moving image.

Furthermore, in the embodiment described above, although it has beendescribed that the target image storing unit 51 and the existing imagestoring unit 52 are included in the storing unit 19 of the imagecapturing apparatus 1, the present invention is not limited to this. Forexample, the target image storing unit 51 and the existing image storingunit 52 can be included in an external server, and the target image dataand the existing image data can be input therefrom and output theretovia the communication unit 20 and the Internet.

Furthermore, in the embodiment described above, it has been describedthat the date comparing unit 65 determines whether or not timedifference between the capture dates of the existing image dataretrieved by the similar image search unit 63 and the target image datafrom which a face region is detected by the face detection unit 61, iswithin the specified time interval.

In the embodiment described above, the specified time interval has beenset to 2 years or 10 years, but the specified time interval is notlimited to this and can be arbitrarily set by a user, as describedabove.

However, the specified time interval determined by the date comparingunit 65 depends on the accuracy of the age recognition by the agerecognition unit 62. Therefore, when the specified time interval is setto be short, it is preferable to increase the age recognition accuracyof the age recognition unit 62 as high as possible.

Furthermore, in the embodiment described above, it has been describedthat the age confirmation unit 64 determines whether or not the age ofthe subject in the target image recognized by the age recognition unit62 is within a predetermined target age range.

In the embodiment described above, the target age range has been set toa range between 0 and 5 years old, but the target age range is notlimited to this and can be arbitrarily set by a user, as describedabove.

For example, in a case in which it is desired that images of brothers 2years apart, taken when in elementary school days, should not be put inthe same group, it is preferable for a user to set the specified timeinterval used by the date comparing unit 65 to 2 years and the targetage range used by the age confirmation unit 64 to a range between 6 and12 years old. With this, it can be possible to classify images ofbrothers having similar faces, taken when in elementary school days,into different groups of the elder and the younger.

Furthermore, in the embodiment described above, although a single agerange has been fixedly employed, the present invention is not limited tothis, and a plurality of age ranges combined or switched as appropriatecan be employed.

Furthermore, in the embodiment described above, although the groupingdetermination processing has been carried out according to the flowchartof FIG. 4, the present invention is not limited to this.

This means that, in the present specification, the steps describing theprogram stored in the storage medium include not only the processingexecuted in a time series following this order, but also processingexecuted in parallel or individually, which is not necessarily executedin a time series.

More specifically, for example, the process of step S16 of confirming anage by the age confirmation unit 64 is not necessarily executed in theorder of FIG. 2.

Furthermore, by omitting the process of step S16 of confirming age bythe age confirmation unit 64, it becomes possible for the grouping unit66 to classify target images including subjects of all ages afterdetermining whether or not the time difference between the capture datesof the target image data and the similar image data is within thespecified time interval.

Furthermore, in the embodiment described above, it has been describedthat the grouping determination processing is terminated if no moresimilar image data is found by the similar image search unit 63 in dataof a plurality of existing images stored in the existing image storingunit 52.

In this case, although the target image data has been stored in theexisting image storing unit 52 without being classified to any group,the present invention is not limited to this, as described above, andthe target image data can be stored in the existing image storing unit52 after being classified into a newly created group.

Furthermore, in the embodiment described above, it has been describedthat the grouping determination processing is terminated if no faceregion is detected by the face detection unit 61 due to a reason such asthere being no subjects included in the target image.

In this case, although the target image data has been stored in theexisting image storing unit 52 without being classified to any group,the present invention is not limited to this, as described above, andthe target image data can be stored in the existing image storing unit52 after being classified into a newly created group.

Furthermore, in the abovementioned embodiment described above, the imageprocessing apparatus according to the present invention is configured byan image capturing processing apparatus such as a digital camera.However, the present invention is not limited to an image capturingprocessing apparatus and can be applied to any electronic device thatcan carry out the image processing described above regardless of whetherthe device has or has not an image capturing function (the target imagedata may be acquired from another device). More specifically, thepresent invention can be applied to a personal computer, a video camera,a portable navigation device, a portable game device, and the like.

The series of processes described above can be executed by hardware andalso can be executed by software.

In a case in which the series of processes are to be executed bysoftware, a program configuring the software is installed from a networkor a storage medium into a computer or the like. The computer may be acomputer embedded in dedicated hardware. Alternatively, the computer maybe capable of executing various functions by installing variousprograms, i.e., a general-purpose personal computer, for example.

The storage medium containing the program may be constituted by theremovable media 31 of FIG. 1. Also, the storage medium containing theprogram may be constituted by a storage medium or the like supplied tothe user in a state incorporated in the device main body in advance. Theremovable media is composed of a magnetic disk (including a floppydisk), an optical disk, a magnetic optical disk, or the like, forexample. The optical disk is composed of a CD-ROM (Compact Disk-ReadOnly Memory), a DVD (Digital Versatile Disk), or the like. The magneticoptical disk is composed of an MD (Mini-Disk) or the like. The storagemedium, supplied in a state in which it is incorporated in the devicemain body in advance, may include the ROM 12 of FIG. 1 in which theprogram is stored, a hard disk included in the storing unit 19 of FIG.1, and the like, for example.

What is claimed is:
 1. An image processing apparatus, comprising: a facedetection unit that detects a region including a face of a subject in atarget image, as a face region, using data of a captured image that canidentify a capture date thereof, as data of the target image; an agerecognition unit that recognize an age of the subject in the targetimage based on the face region detected by the face detection unit; asimilar image search unit that searches for data of an existing imagehaving a face region similar to the face region detected by the facedetection unit, as data of a similar image, from among data of aplurality of existing images that can respectively identify capturedates thereof, each of which belongs to one of a plurality of groups; anage confirmation unit that confirms whether or not the age of thesubject in the target image recognized by the age recognition unit iswithin a predetermined first range; a date comparing unit that, in acase in which the age confirmation unit confirms that the age of thesubject in the target image is within the first range, acquires adifference between a first capture date of the data of the target imageand a second capture date of the data of the similar image found by thesimilar image search unit, and compares the difference with apredetermined second range; and a grouping unit that classifies the dataof the target image into the same group as the group to which the dataof the similar image belongs, in a case in which the difference iswithin the second range according to a result of a comparison by thedate comparing unit.
 2. The image processing apparatus according toclaim 1, wherein in a case in which the difference is outside the secondrange according to the result of the comparison by the date comparingunit, (i) processing carried out by the similar image search unit thatsearches for data of another similar image and the date comparing unitthat acquires the difference using a capture date of data of the othersimilar image as the second capture date and compares the differencewith the second range is repeated until the difference falls within thesecond range, or until there is no more data of an existing image foundas another similar image data by the similar image search unit, and (ii)the grouping unit does not allow classification of the data of thetarget image, or creates a new group and classifies the data of thetarget image into the new group, in a case where there is no moreexisting image data retrieved by the similar image search unit as dataof another similar image.
 3. The image processing apparatus according toclaim 1, wherein the grouping unit classifies the data of the targetimage into a group other than the group to which the data of theexisting image, including subjects of ages within the first range,belongs, in a case in which it is confirmed by the age confirmation unitthat the age of the subject in the target image is outside the firstrange.
 4. The image processing apparatus according to claim 1, whereinthe grouping unit does not allow classification of the data of thetarget image, or creates a new group and classifies the data of thetarget image into the new group, in a case where the face detection unitdoes not detect the face region.
 5. An image processing method carriedout by an image processing apparatus to execute image processing byusing data of a target image as a target, the method comprising:detecting a region including a face of a subject in the target image, asa face region, using data of a captured image that can identify acapture date thereof, as the data of the target image; recognizing anage of the subject in the target image based on the detected faceregion; searching for data of an existing image having a face regionsimilar to the detected face region detected, as data of a similarimage, from among data of a plurality of existing images that canrespectively identify capture dates thereof, each of which belongs toone of a plurality of groups; determining whether or not the recognizedage of the subject in the target image is within a predetermined firstrange; acquiring, in a case in which it is determined that the age ofthe subject in the target image is within the first range, a differencebetween a first capture date of the data of the target image and asecond capture date of the data of the similar image, and comparing thedifference with a predetermined second range; and classifying the dataof the target image into the same group as the group to which the dataof the similar image belongs, in a case in which the difference iswithin the second range according to a result of the comparison.
 6. Anon-transitory storage medium having stored therein a program executableby a computer that controls an image processing apparatus to carry outimage processing on data of a target image as a target, the programcausing the computer to perform functions comprising: detecting a regionincluding a face of a subject in the target image, as a face region,using data of a captured image that can identify a capture date thereofas the data of the target image; recognizing an age of the subject inthe target image based on the detected face region; searching for dataof an existing image having a face region similar to the detected faceregion, as data of a similar image, from among data of a plurality ofexisting images that can respectively identify capture dates thereof,each of which belongs to one of a plurality of groups; determiningwhether or not the recognized age of the subject in the target image iswithin a predetermined first range; acquiring, in a case in which it isdetermined that the age of the subject in the target image is within thefirst range, a difference between a first capture date of the data ofthe target image and a second capture date of the data of the similarimage, and comparing the difference with a predetermined second range;and classifying the data of the target image into the same group as thegroup to which the data of the similar image belongs, in a case in whichthe difference is within the second range according to a result of thecomparison.